shinydo

Dr Rudy Pastel

27 February, 2024

shinydo is the path

Away from painful scripting toward shiny engineering

Only the craftsman can walk the path

Key ideas

  1. R is functional: use functions and shiny modules.
  2. R is software: use version control and packages.
  3. Teamwork needs processes: use Golem.

A curriculum concludes this presentation

R, if you did not know

R is a first choice programming language for:

  • data analysis via statistics, machine learning, mining …
  • visualisation of tables, maps, networks …
  • publication of reports, book, reproducible research …

Shiny brings the analysis results to users via a gui.

Individual contribution

The boss tasks you with developing a Google trend index analysis. You submit the following shiny app.

shinydo::runInternalApp(
  appName = 'googleTrendIndex_V0', 
  display.mode = 'showcase'
)

What do you think of it?

My take: small is beautiful

Because the app is tiny, the code seems acceptable.

  • The code is divided in sections with clear purposes
  • The code is readable due to spanning many short lines
  • Comments are plentiful and informative
  • Variable names are descriptive

None the less, its structure will not survive extension.

Google, Bing, and Yahoo

The boss now wants you to include Bing and Yahoo.

How do you do it?

shinydo::pauseAndThinkAboutIt(seconds = 15)
message('Share your ideas via the common chat.')

Copy, paste, rename ?

shinydo::runInternalApp(
  appName = 'googleTrendIndex_V1', 
  display.mode = 'showcase'
)

My take: Do not do that!

  • Unstructured length begets repulsion and tedium.
  • Repetition kills readability and maintainability.
  • Intertwining business and shiny processes confuses.

Such code usually dies off in abandon. But not always…

Top 10 search engines analysis

The boss now wants a top 10 analysis.

How do you do it?

shinydo::pauseAndThinkAboutIt(seconds = 15)
message('Share your ideas via the common chat.')

Good ideas

  • Negotiate to first provide a top 5 to buy time
  • Restructure the code and enable future extensions
  • Leverage the functional nature of R
  • Separate frontend, backend and support functions

Good organisation

#> C:/Users/rudyp/AppData/Local/R/win-library/4.3/shinydo/apps/googleTrendIndex_V2
#> ├── app.R
#> ├── data
#> │   └── trendData.csv
#> └── R
#>     ├── indexTrendModule.R
#>     └── plotTrendIndex.R

Good implementation

  • Define the frontend in ui.R
  • Define the backend in server.R
  • Define frontend and backend in app.R if trivial
  • Define the functions in the R/ folder
  • Implement the business logic in pure R functions
  • Implement the shiny logic as shiny modules

Shiny modules structure apps

  • A Shiny module encapsulates Shiny logic
  • A Shiny module avoids code repetition
  • A Shiny module can be tested
  • A Shiny module can be nested
  • A Shiny module is a pair of functions
    • One function defines a ui building block
    • One function defines the associated server logic

Shiny module demo

shinydo::runInternalApp(
  appName = 'shinyModuleDemo', 
  display.mode = 'showcase'
)
  • exampleModuleUI and exampleModuleServer define the module
  • The module is called twice
    • Once with identifier examplemodule1
    • Once with identifier examplemodule2.

Shiny module demo code : organisation

#> C:/Users/rudyp/AppData/Local/R/win-library/4.3/shinydo/apps/shinyModuleDemo
#> ├── app.R
#> ├── R
#> │   ├── example-module.R
#> │   └── example.R
#> └── tests
#>     ├── shinytest
#>     │   └── mytest.R
#>     ├── shinytest.R
#>     ├── testthat
#>     │   ├── test-examplemodule.R
#>     │   ├── test-server.R
#>     │   └── test-sort.R
#>     └── testthat.R

Shiny module demo code : ui

exampleModuleUI <- function(id, label = "Counter") {
  # All uses of Shiny input/output IDs in the UI must be namespaced,
  # as in ns("x").
  ns <- NS(id)
  tagList(
    actionButton(ns("button"), label = label),
    verbatimTextOutput(ns("out"))
  )
}

Shiny module demo code : server

exampleModuleServer <- function(id) {
  # shiny::moduleServer() wraps a function
  # to create the server component of a module.
  moduleServer(
    id,
    function(input, output, session) {
      count <- reactiveVal(0)
      observeEvent(input$button, {
        count(count() + 1)
      })
      output$out <- renderText({
        count()
      })
      count
    }
  )
}

Top 5 search engine trend index

shinydo::runInternalApp(
  appName = 'googleTrendIndex_V2', 
  display.mode = 'showcase'
)

The top 5 app structure is so crystal clear, there is no need to split its code between ui.R and server.R

plotTrendIndex

plotTrendIndex = function(date, close, smoother, smootherSpan){
    # Plot the trend index
    color = "#434343"
    par(mar = c(4, 4, 1, 1))
    plot(x = date, y = close, type = "l",
         xlab = "Date", ylab = "Trend index",
         col = color, fg = color, col.lab = color,
         col.axis = color)
    # Optionally, display smoother
    if(smoother){
        smooth_curve <- lowess(x = as.numeric(date),
                               y = close, f = smootherSpan)
        lines(smooth_curve, col = "#E6553A", lwd = 3)
    }
}
  • The plot function is pure R
  • The plot function can be developed, tested and reused independently

indexTrendModule.R

  • The module code is as simple as the very first app.
  • No nested module required
indexTrendModuleUI <- function(id, searchEngine, choices) {
    # All uses of Shiny input/output IDs in the UI must be namespaced,
    # as in ns("x").
    ns <- NS(id)
    tagList(
        titlePanel(sprintf(fmt = "%s Trend Index", searchEngine)),
        sidebarLayout(
            sidebarPanel(

                # Select type of trend to plot
                selectInput(inputId = ns("type"),
                            label = strong("Trend index"),
                            choices = choices,
                            selected = "Travel"),

                # Select date range to be plotted
                dateRangeInput(inputId = ns("date"),label =  strong("Date range"),
                               start = "2007-01-01", end = "2017-07-31",
                               min = "2007-01-01", max = "2017-07-31"),

                # Select whether to overlay smooth trend line
                checkboxInput(inputId = ns("smoother"),
                              label = strong("Overlay smooth trend line"),
                              value = FALSE),

                # Display only if the smoother is checked
                conditionalPanel(
                    condition = sprintf(fmt = 'input["%s"] == true', ns("smoother")),
                    sliderInput(inputId = ns('smootherSpan'), label = "Smoother span:",
                                min = 0.01, max = 1, value = 0.67, step = 0.01,
                                animate = animationOptions(interval = 100)),
                    HTML("Higher values give more smoothness.")
                )
            ),

            # Output: Description, lineplot, and reference
            mainPanel(
                plotOutput(outputId = ns("lineplot"), height = "300px"),
                textOutput(outputId = ns("desc"))
            )
        )
    )
}


indexTrendModuleServer <- function(id, trend_data) {
    # moduleServer() wraps a function to create the server component of a
    # module.
    moduleServer(
        id,
        function(input, output, session) {
            # Subset data
            selected_trends <- reactive({
                req(input$date)
                validate(need(!is.na(input$date[1]) & !is.na(input$date[2]), "Error: Please provide both a start and an end date."))
                validate(need(input$date[1] < input$date[2], "Error: Start date should be earlier than end date."))
                trend_data %>%
                    filter(
                        type == input$type,
                        date > input$date[1] & date < input$date[2]
                    )
            })

            # Create scatterplot object the plotOutput function is expecting
            output$lineplot <- renderPlot({
                plotTrendIndex(
                    date = selected_trends()$date,
                    close = selected_trends()$close,
                    smoother = input$smoother,
                    smootherSpan =  input$smootherSpan)
            })

            # Pull in description of trend
            output$desc <- renderText({
                'The data is a randomly generated place holder.'
            })
        }
    )
}

Top 10 search engine trend index

Extending the app to a top 10 is now trivial.

shinydo::runInternalApp(
  appName = 'googleTrendIndex_V3', 
  display.mode = 'showcase'
)

The boss is happy and praises you. You bask in glory.

Sharing is caring

Colleagues want to build on your analysis and app.
The boss orders you to share.

How do you do it?

shinydo::pauseAndThinkAboutIt(seconds = 15)
message('Share your ideas via the common chat.')

Please, do not

  • Share code via mail or folder
  • Assume users will make it work alone
  • Organise “quick” explain and install meetings
  • Organise even “quicker” bug fix meetings

Instead, do this

  • Convert your code into Write a R package
  • Store the package on a repository
  • Direct users to the repository
  • When needed, update the package in the repo

Simple for users

# Get started
install.packages('shinydo')
vignette(package = 'shinydo', 'shinydo')
shinydo::showPresentation()

# Run the app
shinydo::startTrendIndexDashboard ()

# Query documentation
?shinydo::getTrendIndexData
?shinydo::plotTrendIndex

# Use key functions
trend_data = shinydo::getTrendIndexData(searchEngine = 'Google')
trend_data = subset(x = trend_data, subset = type == 'A')
shinydo::plotTrendIndex(date = trend_data$date, 
                        close = trend_data$close, 
                        smoother = TRUE, 
                        smootherSpan = 0.3)

R packages 101

  1. Create the package skeleton
  2. List required packages in DESCRIPTION
  3. Implement and document functions in R/
packageFolder = file.path(tempdir(), 'demoPackage')
devtools::create(path = packageFolder, open = FALSE)
#> C:\Users\rudyp\AppData\Local\Temp\RtmpOwX7BC/demoPackage
#> ├── DESCRIPTION
#> ├── NAMESPACE
#> └── R

A DESCRIPTION file

{r,echo=TRUE,eval=FALSE,cap="DESCRIPTION"} Package: shinydo Title: A Craftsman's Journey Toward Shiny Mastery Version: 0.0.4 Authors@R: c(person(given = "Rudy", family = "Pastel", role = c("aut", "cre"),, email = "rudy.pastel@gmail.com")) Description: Shinydo is a narrative guiding the Shiny craftsman from "Hello World" to Shiny software engineering. License: `use_gpl3_license()` Imports: shiny (>= 1.6.0), shinythemes (>= 1.2.0), dplyr (>= 1.0.5), readr (>= 1.4.0) Suggests: testthat (>= 3.0.2), devtools (>= 2.3.2), knitr (>= 1.30), rmarkdown (>= 2.8), usethis (>= 2.0.1), DiagrammeR (>= 1.0.6.1), fs (>= 1.5.0) Encoding: UTF-8 LazyData: true Roxygen: list(markdown = TRUE) RoxygenNote: 7.2.3 VignetteBuilder: knitr Config/testthat/edition: 3

A documented function

#' Plot trend index
#'
#' Plot trend index as a time series and optionally a smoothed version.
#'
#' @param date A `Date` vector of same length as `close`.
#' @param close Index value at close time as a `numeric` vector of same length as `date`.
#' @param smoother Should a [stats::lowess()] smoothed version of the trend index
#' be plotted as well? Either `TRUE` and `FALSE`.
#' @param smootherSpan the smoother span. This gives the proportion of points in
#' the plot which influence the smooth at each value. Larger values give more
#' smoothness.
#'
#'
#' @return Nothing silently.
#' @export
plotTrendIndex = function(date, close, smoother, smootherSpan){
    # Plot the trend index
    color = "#434343"
    par(mar = c(4, 4, 1, 1))
    plot(x = date, y = close, type = "l",
         xlab = "Date", ylab = "Trend index", col = color, fg = color, col.lab = color, col.axis = color)
    # Optionally, display smoother
    if(smoother){
        smooth_curve <- lowess(x = as.numeric(date), y = close, f = smootherSpan)
        lines(smooth_curve, col = "#E6553A", lwd = 3)
    }
}

A function documentation

Documentation of plotTrendIndex

Test 101

  1. Manually call functions with random inputs
  2. Design test scenarios
  3. Automatically run test scenarios
# Be in the project directory
setwd(packageFolder)
# creates and opens R/myNewFunction.R
usethis::use_r(name = "myNewFunction")   
# creates and opens tests/testthat/test-myNewFunction.R
usethis::use_test(name = "myNewFunction") 
#> C:\Users\rudyp\AppData\Local\Temp\RtmpOwX7BC/demoPackage
#> ├── DESCRIPTION
#> ├── NAMESPACE
#> ├── R
#> │   └── myNewFunction.R
#> └── tests
#>     ├── testthat
#>     │   └── test-myNewFunction.R
#>     └── testthat.R

A simple test

test_that("multiplication works", {
    expect_equal(2 * 2, 4)
})

test_that("myNewFunction works", {
    expect_equal(myNewFunction(wellChosenInput), expectedOutput)
    expect_error(myNewFunction(wrongInput), 'Error message')
})

Run tests with devtools::test()

R packages 102

  1. Build the documentation
  2. Build the package file
devtools::document()
devtools::build()

R packages 103

  • Store package in a repository such as github.
  • Deploy the app on a server or RStudio Connect

Shiny package 101

Shiny modules and an app starting function

#' Get ui and server functions
#'
#' Build a ui & server pair for [shiny::shinyApp()].
#'
#' @param theme Name of a Shiny theme as listed by [shinythemes::shinythemes].
#' @template searchEngines
#'
getUi = function(searchEngines, theme = "lumen"){
    fluidPage(theme = shinytheme(theme = theme),

              lapply(
                  X = searchEngines,
                  FUN = function(searchEngine, choices){
                      indexTrendModuleUI(id = searchEngine,
                                         searchEngine = searchEngine,
                                         choices = choices)
                  },
                  choices = unique(generateRandomTrendIndexData()$type)
              )

    )
}

#' @rdname getUi
getServer = function(searchEngines){
    function(input, output) {
        for (searchEngine in searchEngines){
            indexTrendModuleServer(
                id = searchEngine,
                trend_data = getTrendIndexData(searchEngine = searchEngine)
            )
        }
    }
}

#' Start the trend index dashboard
#'
#' Explore the trend index data for various search engines
#'
#' @template searchEngines
#' @inheritParams shiny::shinyApp
#'
#' @export
#'
startTrendIndexDashboard = function(
    searchEngines = getSearchEngineNames(),
    onStart = NULL,
    options = list(),
    uiPattern = "/",
    enableBookmarking = NULL){

    shiny::shinyApp(
        ui = getUi(searchEngines = searchEngines),
        server = getServer(searchEngines = searchEngines),
        onStart = onStart,
        options = options,
        uiPattern = uiPattern,
        enableBookmarking = enableBookmarking
    )

}

Main benefits of R packages

  • This standard is a well established and documented
    • Tools, processes and conventions save time
    • Users expect and are used to packages
  • The package enables many quality improving features
    • Documentation of functions and tutorial for usages
    • Test of functions, including shiny modules
    • Reproducible research / data analysis

Teamwork

The boss orders you to lead a team to extend the tool.

How do you organise development?

shinydo::pauseAndThinkAboutIt(seconds = 15)
message('Share your ideas via the common chat.')

Hire the relevant expertise

  • Architecture
  • Cyber-security
  • Dev Ops
  • User experience / User interface

The IT department can support you.

A focus on processes

A software product emerges from the systemic interactions between developers, processes, tools, code and users

The Golem framework is process template

Design

  1. Understand the users’ need and document it
  2. Sketch the user interface with users
  3. Style the user interface with users
  4. Draw the architecture

Prototype

  1. Prototype the ui with a place holding back-end
  2. Prototype the business logic
  3. Users should test and approve in simple cases

Build

  1. Keep business logic, shiny logic and style apart
  2. Define processes w.r.t. to version control, development and tests
  3. Build the Ui and gather user approval
  4. Implement, document and test the business logic
  5. Combine business and shiny logic into modules

Strengthen

  1. Test in a reproducible environment
  2. Test automatically thanks to continuous integration
  3. Test business logic, interactivity, visualisations …
  4. Test speed, memory consumption, load resistance …

Deploy

  1. Decide how you share:
    • Code
    • Package
    • Shiny server access
  2. Design, implement and automate the delivery processes

Questions

R curriculum

Beginners, power users and software developers will find here a list of references to build, hone and professionalise their R and software development skills.

R Language

Package development

Reporting and literate programming

Visualisation

Dashboard

Communities

Other sources of information